test(discovery): render the per-run scorecard - #66
Closed
pengyuzhang wants to merge 1 commit into
Closed
Conversation
One page per run. The aggregate number is for tracking; the individual rows are what somebody fixes, so every false positive and every false negative is listed by manifest id with the evidence the collector recorded. That evidence is the point. The snapshot names the probe, the channel, the path and the rule that fired for each asset, which means a miss can be localized to a probe from the output alone - and a reader should not have to open two JSON files side by side to find that out. Self-contained by construction: one file, no external assets, no network. A scorecard that needed a CDN would be unreadable on the isolated host that produced it. Everything it is handed is escaped, and a test asserts no planted canary value can reach the page - a value redacted out of the snapshot and copied into the report has still left the machine.
lghupan
approved these changes
Aug 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #65.
One HTML page per run, written next to
score.json.The rows are the product, not the numbers
The aggregate is for trend tracking. Every false positive and false negative is listed individually by manifest id, with the evidence the collector recorded — probe, channel, path, and the rule that fired.
That last field is what turns a false positive into a bug report. The snapshot already names why each asset was believed; a reader should not have to open two JSON files side by side to find it.
The page also carries what an aggregate cannot: the baseline asset count, per-field accuracy with the specific entries that were wrong, the excluded entries and why each left the denominator, the canary verdict, and the review-queue result.
Self-contained by construction
One file, no external assets, no network, no script tags. A scorecard that needed a CDN would be unreadable on the isolated host that produced it. It renders in light and dark.
Two things it must not do
Render what it was handed. Everything is escaped; a test feeds it
<img src=x onerror=...>as a run id.Leak a canary. A test walks every planted value and asserts none reaches the page. A value redacted out of the snapshot and copied into a document that then gets shared has still left the machine.
Verification